The present invention relates generally to systems and methods for transmitting and receiving data. More particularly, the present invention relates to systems and methods for transmitting and receiving data that allow data transmissions to be prioritized, that allow dynamic grouping of data recipients, that allow the size of data packets to be varied from file to file, and that provide a manifest process to insure the presence, identity, and integrity of transmitted data files.
Some systems and methods for transmitting and receiving data permit the transmission of data to be prioritized. In many of these systems and methods, however, the data transmissions must all be prioritized before any data transmission can begin in order to insure that transmission priorities are maintained. For example, under these systems and methods, once a data transmission has begun for a low priority data file, a data transmission for a high priority data file must wait for the low priority transmission to complete before the high priority transmission can begin. In many cases this approach to prioritization results in bottlenecking of data where very large, low priority data files that are being transmitted prevent the transmission of many small, high priority data files. Similarly, bottlenecking can also occur in instances where many large, high priority data files prevent a small, low priority file from being transmitted in a timely fashion.
Frequently it is desirable to send a single data transmission from a transmitter to multiple recipients. Some systems and methods for transmitting data allow such transmissions to be performed to predetermined groups of individuals. Some of these and other systems and methods for transmitting data allow recipients to receive data for groups in which the recipients are interested. None of these systems and methods, however, allows a transmitter to dynamically specify the recipients for a single transmission at the time of the transmission.
Many systems and methods for transmitting data use packet based transmission to transmit data from one place to another. These forms of packet based transmission, however, are rigid in that they fix the size of every packet regardless of the data file type, the data file size, or the characteristics of the path over which the data is to be transmitted. Thus, in many of these systems and methods, the same packet size may be used to transmit a 100 byte file as is used to transmit a 100 Mega-byte file, for example. Using the same size packet regardless of the size of the file to be transmitted is inefficient because the percentage of the file in each packet shrinks rapidly as the size of the transmitted file grows.
Finally, in many systems and methods for transmitting data, groups of data files are transmitted from one place to another as bundles of data files. Unfortunately, however, once these files arrive at their destination, the batch of files must be presumed to include all of the intended files because no additional indicator is provided as to what files should be present. Additionally, each of the transmitted files is usually only identifiable by the name, date, and time associated with the file. Lastly, in order to process the received files properly, the receiving system must know what to do with the files after they have arrived.
In view of the foregoing, it would be desirable to provide a system and method for transmitting and receiving data in which data being transmitted can be prioritized while not causing bottlenecking of the data.
It would also be desirable to provide a system and method for transmitting and receiving data in which multiple recipients of data can be dynamically specified just prior to a group data transmission.
It would be further desirable to provide a system and method for transmitting and receiving data in which the size of packets used in the transmission of data can be varied from file to file.
It would be even further desirable to provide a system and method for transmitting and receiving data which can verify the presence, identity, and integrity of each data file in a bundle of transmitted data files.
It is therefore an object of this invention to provide a system and method for transmitting and receiving data in which data being transmitted can be prioritized while not causing bottlenecking of the data.
It is another object of this invention to provide a system and method for transmitting and receiving data in which multiple recipients of data can be dynamically specified just prior to a group data transmission.
It is a further object of this invention to provide a system and method for transmitting and receiving data in which the size of packets used in the transmission of data can be varied from file to file.
It is a further object of this invention to provide a system and method for transmitting and receiving data which can verify the presence, identity, and integrity of each data file in a bundle of transmitted data files.
These and other objects of the present invention are achieved by providing a system and method for transmitting and receiving data that allow data transmissions to be prioritized, that allow dynamic grouping of data recipients, that allow the size of data packets to be varied from file to file, and that provide a manifest process to insure the presence, identity, and integrity of transmitted data files.
In illustrative embodiments of the present invention, the data transmission and reception capabilities of the system and method are provided by a data transmission facility, at least one forward data path, at least one data reception facility, and at least one reverse data path. Generally speaking, data is transmitted in this system and method in response to a request to transmit data. This request may be locally generated at the data transmission facility, or may be remotely generated at any of the data reception facilities and transmitted to the data transmission facility through the reverse data path. Once a request has been received, the requested data is then prepared and transmitted from the data transmission facility through at least one forward data path to at least one data reception facility. In the event that errors are detected in this transmission of data, retransmissions of the corrupted data may then be requested by any of the data reception facilities through the reverse data path. The transmitted data is finally received by at least one data reception facility, where a copy of the requested data file is recovered and the data is used as desired.
Data may be transmitted in the form of at least one header packet and at least one data packet for each data file. The one or more header packets each include information to identify the associated data file, to identify the one or more target data reception facilities, to identify and describe the associated data packets, to indicate the data file transmission priority, and to indicate the integrity of each header packet and data file. Each data packet transmitted in association with a data file includes an associated header packet identifier, a unique data packet identifier, a data packet size indicator, at least a portion of the data from the data file, and a code to check the integrity of the data in the data packet.
Prior to transmission, each of these header and data packets is constructed and queued at the data transmission facility. In response to a remote procedure call (RPC) request that is received at the data transmission facility, an RPC server constructs a new file header packet for new transmission requests or retrieves an existing file header packet from storage for retransmission requests. This file header packet is then transferred to one of a number of data transmission queues based upon the size of the requested file and the priority indicated in the RPC request. Once the header packet has been transferred to a data transmission queue, the RPC server waits for and processes other RPC requests. After receiving this file header packet, each data transmission queue notifies a scheduler that the queue has a data file that is waiting to be transmitted.
Whenever any of the queues contains data waiting to be transmitted, the scheduler continuously selects the next queue from which a packet is to be transmitted and the forward data path on which that queue is to do so. In selecting this next queue, the scheduler preferably attempts to maintain, based upon the priority of each queue, a fixed ratio of the number of packet transmissions for each queue as compared to the total number of packets transmitted in all of the queues. For example, in a data transmission facility with five queues that have priorities of xe2x80x9c1,xe2x80x9d xe2x80x9c2,xe2x80x9d xe2x80x9c3,xe2x80x9d xe2x80x9c4,xe2x80x9d and xe2x80x9c5,xe2x80x9d wherein priority xe2x80x9c1xe2x80x9d is the highest priority and priority xe2x80x9c5xe2x80x9d is the lowest priority, the scheduler may attempt to maintain the number of packet transmissions at sixteen, eight, four, two, and one for each of the five queues, respectively, for every thirty one total packets transmitted from all of the queues combined. In selecting the forward data path on which each queue is to transmit a header or data packet, the scheduler preferably attempts to direct each queue to the fastest available forward data path that is not being used when each queue is given permission to transmit.
Once permission and a forward data path selection have been received from the scheduler at a data transmission queue, the data transmission queue retrieves and transmits on the selected forward data path, the next header or data packet that is to be transmitted. After transmitting a header or data packet, the data transmission queue once again waits for a new permission indication and a forward data path selection until all of the data packets for a data file have been transmitted. Upon all of the data packets being transmitted, the data transmission queue then notifies the scheduler that no file is waiting to be transmitted so that the scheduler will not attempt to give the queue permission to transmit data. The queue then copies, if necessary, the file header packet to storage so that the queue will be able to satisfy any future retransmission requests and then removes the file header packet from the queue. Once the file header packet has been removed from the queue, the queue waits for other file header packets to be received from the RPC server.